Method, device and computer-readable recording medium containing program for extracting object region of interest

ABSTRACT

An object region extracting process for extracting an object region of interest from an image is automated to the maximum possible extent to improve user-friendliness. In this process, an arbitrary point is set in the object region of interest, and a presence area, which is likely to contain the entire object region of interest, is determined in the image using the set arbitrary point and a possible size of the object region of interest. Then, the object region of interest is extracted from the image based on the set arbitrary point and at least one point outside the determined presence area.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a method, a device and acomputer-readable recording medium containing a program for extractingan object region of interest from an image, and in particular to amethod, a device and a computer-readable recording medium containing aprogram for extracting an object region of interest, such as a lesionregion or an organ region from a medical image.

2. Description of the Related Art

In the medical field, a process of extracting and displaying a certainobject region of interest, such as a lesion region or an organ region,from a medical image has conventionally been conducted to provide animage with high diagnosis performance.

An example of the method for extracting an object region from an imageis described, for example, in US Patent Application Publication No.20040008886, where the user specifies certain pixels in the imagerepresenting an object region and other certain pixels in the imagerepresenting a background region. Then, a probability indicating whethereach pixel represents the object or the background is calculated basedon information of the marked pixels, and a probability indicatingwhether or not each combination of neighboring pixels are pixelsbelonging to the same region is calculated based on a local densitydifference in the image. Then, the object region is extracted from theimage using these probabilities.

Another example of the method for extracting an object region from animage is proposed in R. Tachibana and S. Kido, “Automatic segmentationof pulmonary nodules on CT images by use of NCI Lung Image DatabaseConsortium”, Proc. of SPIE, Vol. 6144, pp. 61440M-1-61440M-9, 2006 wherethe user specifies certain pixels in the image representing an objectregion and other certain pixels in the image representing a backgroundregion, and these pixels are respectively used as reference points forregion growing for the object region and reference points for regiongrowing for the background region. Then, a probability indicatingwhether or not each combination of neighboring pixels are pixelsbelonging to the same region is calculated based on a local densitydifference in the image. Then, each reference point is subjected toregion growing using the probability, and a boundary is created alongpoints where the grown object region and the grown background regionmeet, to extract the object region from the image.

In the techniques proposed in these documents, however, the user isrequired to manually specify certain pixels representing an objectregion and other certain pixels representing a background region in theimage, and this is troublesome. Therefore, it is desired to reducetroublesomeness of the manual input by the user.

SUMMARY OF THE INVENTION

In view of the above-described circumstances, the present invention isdirected to providing a method, a device and a computer-readablerecording medium containing a program for extracting an object region ofinterest, in which a process of extracting the object region isautomated to the maximum possible extent to improve user-friendliness.

An aspect of the invention is a method for extracting an object regionof interest from an image. The method includes: setting an arbitrarypoint in the object region of interest; determining a presence area inthe image using the set arbitrary point and a possible size of theobject region of interest, the presence area being likely to contain theentire object region of interest; and extracting the object region ofinterest from the image based on the set arbitrary point and at leastone point outside the determined presence area.

Another aspect of the invention is a device for extracting an objectregion of interest from an image. The device includes: setting means toset an arbitrary point in the object region of interest; areadetermining means to determine a presence area in the image using theset arbitrary point and a possible size of the object region ofinterest, the presence area being likely to contain the entire objectregion of interest; and object region extracting means to extract theobject region of interest from the image based on the set arbitrarypoint and at least one point outside the determined presence area.

Yet another aspect of the invention is a computer-readable recordingmedium containing a program for causing a computer to carry out aprocess for extracting an object region of interest from an image. Theprocess includes: setting an arbitrary point in the object region ofinterest; determining a presence area in the image using the setarbitrary point and a possible size of the object region of interest,the presence area being likely to contain the entire object region ofinterest; and extracting the object region of interest from the imagebased on the set arbitrary point and at least one point outside thedetermined presence area.

The object region of interest may be a lesion region in the medicalimage or an organ region in the medical image.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating one embodiment of an objectregion extracting device of the present invention,

FIG. 2 is a diagram for explaining one example of a method fordetermining a presence area by an area determining means shown in FIG.1,

FIG. 3 is diagram for explaining one example of how an object region ofinterest is extracted by an object region extracting means shown in FIG.1,

FIG. 4 is diagram for explaining one example of how the object region ofinterest is extracted by the object region extracting means shown inFIG. 1,

FIG. 5 is a diagram illustrating one example of the object region ofinterest extracted by the object region extracting device of theinvention, and

FIG. 6 is a flow chart illustrating one embodiment of an object regionextracting method of the invention.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Hereinafter, an embodiment of an object region extracting device of thepresent invention will be described with reference to the drawings,where the object region extracting device extracts a lesion region froma two-dimensional medical image. It should be noted that theconfiguration of the object region extracting device 1 shown in FIG. 1is implemented by executing an object region extracting program, whichhas been read in an auxiliary storage device, on a computer (such as apersonal computer). The object region extracting program to be installedon the computer may be stored in an information storage medium, such asa CD-ROM, or may be distributed over a network, such as the Internet.

The object region extracting device 1 extracts a lesion region R from amedical image I, which has been taken with an imaging apparatus such asa CT apparatus. As shown in FIG. 1, the object region extracting device1 includes: a setting means 10 to set an arbitrary point Ps in thelesion region R; an area determining means 20 to determine a presencearea E, within which the entire lesion region R may possibly becontained, in the medical image I using the arbitrary point Ps and apossible size L of the lesion region R; and an object region extractingmeans 30 to extract the lesion region R from the medical image I basedon the arbitrary point Ps and a point outside the presence area E.

The setting means 10 sets the arbitrary point Ps in the lesion region Rcontained in the medical image I which has been taken with an imagingapparatus such as a CT apparatus. The setting means 10 may set thearbitrary point Ps at a position, for example, of a chest noduledetected by using a chest nodule detection method as described in K.Suzuki et al., “Comparison between 2D and 3D massive-training ANNs(MTANNs) in CAD for lung nodule detection on MDCT”, Int J CARS 1, pp.354-357, 2006. Alternatively, the user may specify a position on themedical image I, which is displayed on an image display device, using aposition specifying means, such as a mouse or a keyboard, provided tothe object region extracting device 1, and the position specified by theuser may be set as the arbitrary point Ps.

The arbitrary point Ps is formed by one or more points set in the lesionregion R. The arbitrary point may be set at a rough center of the lesionregion R, or may be set out of the center of the lesion region R.

The area determining means 20 determines the presence area E withinwhich the entire lesion region R may possibly be contained, i.e., anarea that is likely to contain the entire lesion region R, in themedical image I using the arbitrary point Ps and the possible size L ofthe lesion region R. Specifically, the presence area E is determinedsuch that the size of the presence area E is equal to or larger than thepossible size L of the lesion region R, and the rough center of the areais set at the position of the arbitrary point Ps (if there are two ormore arbitrary points Ps, at the center position between the points).

The size of the presence area E is determined to be equal to or largerthan the possible size L of the lesion region R, for example, 1.5 timesthe possible size L of the lesion region R, so that the presence area Ehaving the rough center thereof set at the arbitrary point Ps cancontain the entire lesion region R even when the arbitrary point Ps isset at a position out of the center of the lesion region R.

The possible size L of the lesion region R is a physically-possiblemaximum size of the lesion region. The possible size L of the lesionregion R may be a size (the number of pixels) on the medical image I,which is obtained by dividing the physically possible maximum size ofthe lesion represented by the lesion region R by a size represented byone pixel on the medical image I. Alternatively, the largest one ofsizes of lesion regions contained in a number of medical images may beused as the possible size L of the lesion region R. The possible size Lof the lesion region R may be determined by any other method.

As shown in FIG. 2, for example, on a two-dimensional medical image Icontaining the lesion region R representing a chest nodule, if aphysically possible maximum width of the chest nodule represented by thelesion region R on the medical image I is 30 mm and a size representedby one pixel on the medical image I is 0.5 mm×0.5 mm in the vertical andhorizontal directions, then the maximum width is 60 pixels (30 mm/0.5mm) on the medical image I. Therefore, a width W of the possible size ofthe lesion region R may be determined to be 60 pixels, and the presencearea E may be determined to be a square area of 90×90 pixels (90 pixelsis 1.5 times the width W in this case), with the center of the presencearea E being positioned at the arbitrary point Ps set in the lesionregion.

It should be noted that a peripheral shape of the presence area E maytake any of various shapes, such as a square, a circle or an ellipse.

The object region extracting means 30 extracts the lesion region R fromthe medical image I based on the arbitrary point Ps and a point Ptoutside the presence area E. For example, an area to be determined Dcontaining the presence area E is set on the medical image I. Then,based on the fact that the arbitrary point Ps is a pixel representingthe lesion region R and the one or more points Pt set outside thepresence area E are pixels representing the background region, the areato be determined D is segmented into the lesion region R and thebackground region to extract the lesion region R according to a GraphCut region segmentation method described in Yuri Y. Boykov andMarie-Pierre Jolly, “Interactive Graph Cuts for Optimal Boundary &Region Segmentation of Objects in N-D images”, Proceedings of“International Conference on Computer Vision”, Vancouver, Canada, Vol.I, pp. 105-112, 2001.

In this segmentation method, first, as shown in FIG. 3, a graph iscreated, which includes nodes Nij representing respective pixels in thearea to be determined D, nodes S and T representing labels (the lesionregion R and the background region in this embodiment) that the pixelsmay take, n-links connecting nodes of neighboring pixels to each other,and t-links connecting the nodes Nij representing the pixelsrespectively to the node S representing the lesion region or the node Trepresenting the background region of the lesion. The thickness of eachn-link indicates a probability of each pair of neighboring pixels beingpixels belonging to the same region. The probability is calculated basedon a distance between the neighboring pixels and a difference betweenpixel values of the neighboring pixels.

Each t-link that connects each node Nij representing a pixel to the nodeS representing the lesion region R indicates a probability of the pixelbeing a pixel in the lesion region R, and each t-link that connects eachnode representing a pixel to the node T representing the backgroundregion R indicates a probability of the pixel being a pixel in thebackground region R. If information about whether each pixel belongs tothe lesion region R or the background region has already been given,these probabilities can be set according to the given information. Ifsuch information is not given, the probabilities can be calculated basedon statistic characteristics of pixel values of one or more pixels whichhave been known as being pixels in the lesion region R or in thebackground region.

Since the arbitrary point Ps is a pixel set in the lesion region, athick t-link is set to connect a node N33 representing the point Ps tothe node S representing the lesion region, as shown in FIG. 3. Further,since the points Pt set outside the presence area E are pixelsrepresenting the background region, thick t-links are set to connectrespective nodes N11, N12, . . . , N15, N21, N25, N31 representing therespective points Pt to the node T representing the background region.

Since the lesion region R and the background region are mutuallyexclusive regions, for example, as indicated by dashed lines in FIG. 4,appropriate links among all the t-links and n-links are cut to separateoff the node S from the node T to segment the area to be determined Dinto the lesion region R and the background region. At this time,optimal region segmentation can be achieved by cutting the t-links andthe n-links such that a sum of the probability values of all the linksto be cut becomes minimum. One example of the lesion region R extractedby the above-described region segmentation is shown in FIG. 5. In FIG.5, the contour of the lesion region R is indicated by solid lines.

Now, an object region extracting method of the invention will bedescribed with reference to a flow chart shown in FIG. 6. First, thesetting means 10 sets the arbitrary point Ps in the lesion region in themedical image I (step ST1). Then, the area determining means 20determines the presence area E, within which the entire lesion region Rmay possibly be contained, in the medical image I based on the arbitrarypoint Ps set in step ST1 and the possible size L of the lesion region R(step ST2). Subsequently, the object region extracting means 30 extractsthe lesion region from the medical image I based on the arbitrary pointPs set in step ST1 and the point Pt outside the presence area Edetermined in step ST2 (step ST3).

According to the above-described embodiment, in order to extract anobject region of interest from an image, an arbitrary point is set inthe object region of interest, and a presence area, within which theentire object region of interest may possibly be contained, isdetermined in the image using the set arbitrary point and a possiblesize of the object region of interest. Then, the object region ofinterest is extracted from the image based on the set arbitrary pointand a point outside the determined presence area. In the conventionalobject region extracting methods, the user is required to manuallyspecify certain pixels representing the object region of interest andother certain pixels representing the background region in the image. Incontrast, the point outside the determined presence area is used as apixel representing the background region in this method, which meansthat the operation of manually specifying the pixel representing thebackground region is automated. Therefore, troublesomeness of manualinput is reduced, thereby providing improved user-friendliness.

Although the object region of interest is extracted from thetwo-dimensional image in the above-described embodiment of the objectregion extracting device of the invention, the object region of interestcan also be extracted from a three-dimensional image. For example, thesetting means 10 sets in the object region of interest in thethree-dimensional image an arbitrary point Ps in a three-dimensionalcoordinate system. Then, the area determining means 20 determines athree-dimensional presence area E, within which the entire object regionof interest may possibly be contained, in the image using the arbitrarypoint Ps and a possible size L of the object region of interest. Then,the object region extracting means 30 extracts a three-dimensionalobject region of interest from the image based on the arbitrary point Psand a point Pt outside the presence area E, using the above-describedsegmentation method or any other method. In this case, the peripheralshape of the presence area E may take any of various shapes, such as ahexahedron or a sphere.

It should be noted that, in the object region extracting device of theinvention, the possible size of the object region of interest means aphysically possible maximum size for sizes of the same type of objectregions. In a case where two or more object regions of interest areextracted from an image using the object region extracting device of theinvention, a list of possible sizes of object regions determined forrespective types of object regions may be prepared and referenced toappropriately determine the presence area for each type of the objectregion of interest to be extracted.

It should be noted that the object region may be an organ regionrepresenting an organ, such as liver, spleen or kidney, or a lesionregion representing a lesion, such as brain tumor, chest nodule, livertumor, liver cyst or kidney cyst.

According to the method, the device and the computer-readable recordingmedium containing the program for extracting an object region ofinterest from an image, in order to extract an object region of interestfrom an image, an arbitrary point is set in the object region ofinterest, and a presence area, within which the entire object region ofinterest may possibly be contained, is determined in the image using theset arbitrary point and a possible size of the object region ofinterest. Then, the object region of interest is extracted from theimage based on the set arbitrary point and a point outside thedetermined presence area. In contrast to the conventional object regionextracting methods where the user is required to manually specifycertain pixels representing the object region of interest and othercertain pixels representing the background region in the image, thepoint outside the determined presence area is used as a pixelrepresenting the background region in this method, which means that theoperation of manually specifying the pixel representing the backgroundregion is automated. Therefore, troublesomeness of manual input isreduced, thereby providing improved user-friendliness.

1. A method for extracting an object region of interest from an image,the method comprising: setting an arbitrary point in the object regionof interest; determining a presence area in the image using the setarbitrary point and a possible size of the object region of interest,the presence area being likely to contain the entire object region ofinterest; and extracting the object region of interest from the imagebased on the set arbitrary point and at least one point outside thedetermined presence area.
 2. A device for extracting an object region ofinterest from an image, the device comprising: setting means to set anarbitrary point in the object region of interest; area determining meansto determine a presence area in the image using the set arbitrary pointand a possible size of the object region of interest, the presence areabeing likely to contain the entire object region of interest; and objectregion extracting means to extract the object region of interest fromthe image based on the set arbitrary point and at least one pointoutside the determined presence area.
 3. The object region extractingdevice as claimed in claim 2, wherein the object region of interestcomprises a lesion region in the medical image.
 4. The object regionextracting device as claimed in claim 2, wherein the object region ofinterest comprises an organ region in the medical image.
 5. Acomputer-readable recording medium containing a program for causing acomputer to carry out a process for extracting an object region ofinterest from an image, the process comprising: setting an arbitrarypoint in the object region of interest; determining a presence area inthe image using the set arbitrary point and a possible size of theobject region of interest, the presence area being likely to contain theentire object region of interest; and extracting the object region ofinterest from the image based on the set arbitrary point and at leastone point outside the determined presence area.